Title: Replication File for �Quality Assessment of the Academic Freedom Index: Strengths, Weaknesses, and How Best to Use It�
Journal: Perspectives on Politics
DOI: TBA
Link: TBA
Author: Lars Lott and Janika Spannagel
Correspondence: Lars Lott, Friedrich-Alexander-University Erlangen-N�rnberg (lars.lott@fau.de)
Date: September 19th, 204

Required Software: R and RStudio (4.4.1 or Above). 
Additional required software packages and libraries are specified in the replication code. User should install the respective libraries by their own. 
An additional local library called "vutils" can be installed from the local folder. 

Overview: These files replicate all analyses presented in �Quality Assessment of the Academic Freedom Index: Strengths, Weaknesses, and How Best to Use It�, P
erspectives on Politics. https://doi.org/XXX

To replicate all analyses and figures, download all data files to a folder entitled �~/data�, create folders entitled �~/outputs_original_data� and �~/outputs_simulated_data� in the same directory. 
The R scripts are stored in the folders named "~/scripts_original_data" and "~/scripts_simulated data". Create also an folder named "~/results".

The data folder contains all data that is necessary to computationally reproduce the results presented in this article and the supplementary appendix, except of specific variables of 
V-Dem's post-survey questionnaire (PSQ). Please consider the folder structure in the data folder. 
This PSQ data (including the gender, age, country of residence, government employment, and education level) and named "coder_characteristics_wide.rds" contains potentially identifiable personal 
information and is therefore subject to legal restrictions preventing us from making it public. Every person who would like to reproduce the original results using the PSQ data
(Table 2, Figures 1, 2, 3, 4, 5, and 7) can submit a request to the V-Dem Institute (contact@v-dem.net} for access to the PSQ data. The reproduction materials include scripts using simulated PSQ data to 
show the computationally reproducibility of our results. The findings are stored in the folder named "outputs_simulated_data" and the findings presnted their cannot be substantially interpreted.  

Therefore, scripts or part of these scripts using the original coder characteristics will not be running without this data. 
These scripts are "01_main_analysis.R", "06_main_analyses_perceptions.R", "07_vignettes_analysis.R", 
and " 09_coder_disagreement_in_decline_growth_episodes.R". All other scripts ("02_bfa.R", "03_frequentist_factor_models.R", "04_appendix_correlations_of _rater_errors.R", 
"05_main_analyses_perceptions_data_preprocessing.R", "08_coder_uncertainty.R", "10_posterior_replication_Pelke_2023.R", and "11_posterior_regression_analysis.R") will be runs with the stored data.  

Because we are not allowed to store the original coder characteristics, we simulated these coder characertistics. The file is named "coder_characteristics_wide_simulated_data.rds" by using the file 
"00_simulate_expert_coder_data.R" file stored in the "scripts_simulated_data" folder. "00_simulate_expert_coder_data.R" will not be running as the original coder characteristics data is missing.
Users can run all code stored in the folder "scripts_simulated_data", except of "00_simulate_expert_coder_data.R". 

%%%%%%%%%%%%%% Replication Code for each Table and Figure in Main Paper %%%%%%%%%%%%%%

%%% Section 2.2 Factor analysis of AFI indicators %%%
To reproduce Table 1, run "02_bfa.R". Please consider that depending on the computer power, this scirpt will run multiple hours. On a 8-core mechine with 16BG of memory of took ~24 hours to converge. 
Code was orignally written by Dan Pemstein and V-Dem core team (see McMann et al. 2022 Political Analysis). It has been adapated in this reserch for the Academic Freedom Index.  

%%% Section 3.2  Expert Characteristics and Qualifications %%%
To reproduce Table 2, run "01_main_analysis.R", lines 89 - 255. 

%%% Section 3.4 Index-Level Aggregation %%%
To reproduce Table 3, run "01_main_analysis.R", lines 257- 366.

%%% Section 3.4 Index-Level Aggregation %%%
To reproduce Table 3, run "01_main_analysis.R", lines 257- 366.

%%% Section 3.5 Coverage Across Countries and Time %%%
To reproduce fidnings from this section run "01_main_analysis.R", lines 367 - 388.

%%% Section 3.6 Analyzing Respondent Disagreement %%%
To reproduce Figures 1 and 2, run "01_main_analysis.R", lines 389 - 781.

%%% Section 3.7 Analyzing Individual Respondent Biases %%%
To reproduce Figures 3, 4, and 5, run "01_main_analysis.R", lines 782 - 1294.

%%% Section 4  Convergent Validity Assessment %%%
To reproduce Figures 6 and 7, run "01_main_analysis.R", lines 1295 - 2033.

%%% Section 5 Incorporating Measurement Uncertainty of Latent Variables %%%
To reproduce Figure 8, run "10_posterior_replication_Pelke_2023.R". 

%%%%%%%%%%%%%% Replication Code for each Table and Figure in Appendix %%%%%%%%%%%%%%

%%% Section B  Two-dimensional factor analysis %%%
To reproduce Tables B1 and B2, run "03_frequentist_factor_models.R".

%%% Section C Index-level aggregation %%%
To reproduce Figure C1, run "01_main_analysis.R", lines 257- 366.

%%% Section D Respondent-Correlated Errors %%%
To reproduce Tables D1, D2, and D3, run "04_appendix_correlations_of _rater_errors.R". 
Code was orignally written by Dan Pemstein and collgeues (McMann et al. 2022 Political Analysis) and has been adatped for this project. 

%%% Section E Examining Respondent Disagreement and Biases %%%
To reproduce Tables E1 and E2, and Figures E1 and E2,  run "01_main_analysis.R", lines 389 - 1294.

%%% Section F Convergent Validity Assessment %%%
To reproduce Table F1, and Figures F1 and F2,  run "01_main_analysis.R",  lines 1295 - 2033.

%%% Section G Respondent Disagreement and Biases - Perceptions %%%
To reproduce Tables G1 and G2, and Figures G1, G2, G3, G4, and G5,  run "05_main_analyses_perceptions_data_preprocessing.R" and "06_main_analyses_perceptions_code.R".

%%% Section H Analyzing Anchor Vignettes %%%
To reproduce Table H1 and Figure H1, run "07_vignettes_analysis.R".

%%% Section I Distribution of Coder Confidence %%%
To reproduce Figure I1, run "08_coder_uncertainty.R".

%%% Section J Coder Disagreement in Academic Freedom Growth and Decline Episodes %%%
To reproduce Figue J1, J2, J3, and J4 and Tables J1 and J2, run "09_coder_disagreement_in_decline_growth episodes.R".

%%% Section K Coder Disagreement in Academic Freedom Growth and Decline Episodes %%%
To reproduce Figure K1, run "11_posterior_regression_analysis.R".


%%%%%%%%%%%%%% Data Files (relative path "~/data") %%%%%%%%%%%%%%

%%%% "bfa" folder %%%%
-country_unit.rds
-input includes posterior csv files

%%%% "freedom_house" folder %%%%
-All_data_FIW_2013-2023.xlsx

%%%% "posterior/v13" folder %%%%
-includes all posteriors files named by the respective variables

%%%% "v13" folder %%%%
- coder-characteristics data (empty in the replication files, see above)
- Episodes of Regime Transformation v13 data, stored in "ERT-13" folder
- vdem_13_coder_level: coder-level data with nor sensitive data, publically available at v-dem.net
- vdem_13_cy: country-year V-Dem data
- Episodes of Academic Freedom Growth and Decline: episode_without_uncertainty_interval_test.csv

%%%% "v14" folder %%%%
- coder_scores_post_mm

%%%% stored without seperate folder %%%%
- "all_imputations.rds" replication data for Section K. Is constructed by code "10_posterior_replication_Pelke_2023.R".
- "coder_characteristics_wide_simulated_data.rds": simulated data of the coder_characteristics. Used for running scripts stored in folder "scripts_simulated_data". 
- "coder_ids.rds" coder IDS of AFI coders
- "coder_level_ds_perceptions.rds": perception data, constructed by code "05_main_analyses_perceptions_data_preprocessing.R" and "06_main_analyses_perceptions_code.R".
- "vdem_subset_v13.rds": data with V-Dem v13 data constructed by code "01_main_analysis.R"

%%%%%%%%%%%%%% Output Files (relative path "~/outputs_original_data") %%%%%%%%%%%%%%

Stored tables and figures original data

%%%%%%%%%%%%%% Output Files (relative path "~/outputs_simulated_data") %%%%%%%%%%%%%%

Stored tables and figures simulated data

%%%%%%%%%%%%%% vutils package (relative path "~/vutils") %%%%%%%%%%%%%%

Library built by the V-Dem core team to manage V-Dem data; parts of this library has been used for the analysis presented in this paper. 
Please install this package by installing it from the folder provided in the replication package



